Skip to content

Make let rec build cyclic data, and compare it by bisimulation - #16

Merged
janstrakowski merged 3 commits into
mainfrom
worktree-cyclic-let-rec
Aug 31, 2026
Merged

Make let rec build cyclic data, and compare it by bisimulation#16
janstrakowski merged 3 commits into
mainfrom
worktree-cyclic-let-rec

Conversation

@janstrakowski

Copy link
Copy Markdown
Owner

SPEC.md §6 has always said cycles are possible; nothing could construct one. A let rec over a Table literal now can, so a social graph whose friendships are mutual is expressible — without a placeholder type, a lazy keyword, or a second binding form.

let rec people {
  .alice = { .name = "Alice", .friends = { people.bob, people.carol } },
  .bob   = { .name = "Bob",   .friends = { people.alice } },
  .carol = { .name = "Carol", .friends = { people.alice, people.bob } },
};
people.alice.friends[1].friends[1].name        // => "Alice", back where we started

How

The mechanism is evaluation order, not deferral.

  • The Table is created and bound to the name before any entry runs, so an entry can mention the Table it belongs to.
  • Entries are evaluated on demand rather than in source order. Reaching people.bob while .alice is mid-flight evaluates .bob there and then. That dissolves every dependency between entries that has a topological order — which is why mutual references need nothing further, and why a form binding several names at once is still not needed.
  • What survives is the residual true cycle: .bob reaching back into .alice, already in progress and so with no value to give. Only that yields a forward reference (src/rec_build.odin), filled the moment .alice completes. Storing one makes the back-edge; inspecting one before it is filled is a circular definition and fails, naming the entry. Every entry completes before the let rec returns, so no finished value can hold an unfilled one — which is why this is not one of §3's types.

Scope is deliberately narrow, and the docs say so. This builds cyclic structures — finite graphs with back-edges — not unbounded ones: a stream constructing a fresh node per step still recurses to the nesting budget. Only a Table literal written directly as the bound value has entries to reorder; let rec x x + 1 still reports undefined name.

What then has to cope with a graph rather than a tree

  • Equality is bisimulation. Two rings of the same shape built by separate bindings are the same value — §6 makes equality a question about content, and a pointer comparison would answer it wrongly. Assumptions live in a union-find with path compression (Downey–Sethi–Tarjan congruence closure), keeping it near-linear rather than the quadratic a visited-pair set would cost. The map is created only when two distinct Tables are first assumed equal, so scalar key lookup on the hot path of field access still allocates nothing.
  • Printing labels the node a back-edge returns to#1{n: 1, self: #1} — so output stays finite and the shape is legible. Acyclic values, including a sub-Table merely shared between two branches, print byte-identically to before.

Reviewer notes

Two decisions worth a second opinion:

  1. Hashing a cyclic value is refused, not implementedsha256 of one fails cleanly rather than hanging, alongside the existing directory-File and Function gaps. §3 pins what a digest encodes, so choosing a cyclic encoding is a spec decision. SPEC.md §6 records what the answer looks like (SCC decomposition, Merkle-fold the acyclic part, canonicalise each component under bisimulation) and LANGUAGE.md lists it as unbuilt. This is the one part of the feature I'd call unfinished.
  2. Evaluation order inside a rec Table literal is now dependency order, not source order. It falls out of the design, but it is a "what fails and how" change: where entries have effects — a createfile, a fired async, a failing check — that is the order they happen in, and which failure surfaces first can differ from the source reading. Documented in §10.

The second commit is an audit, not just this feature's loose ends. Three of its six fixes were wrong before this change: §5 claimed a Table's hash sorts entries by §6's generic total order (it sorts by key digest, deliberately, since that ordering isn't built); §5 filed hashing and equality under one heading, reading as though equality were hash-derived when it is structural; and §8 named the nesting constant MAX_EVAL_DEPTH when it has been MAX_NEST_DEPTH throughout.

Testing

  • 180 tests pass, 20 new in src/rec_build_test.odin covering the cycle, demand reordering, source-order preservation, both circular-definition failures, bisimulation (equal, unequal, and differing-period cases), labelled printing, and the hash refusal.
  • New example examples/cyclic-data.hb, asserted by the suite like every other.
  • Every snippet added to LANGUAGE.md and every behavioural claim added to SPEC.md was run against hb before being written down.
  • The portable WASI target builds clean, so the new code compiles for wasm32. The threaded WASI build needs clang and the smoke test needs wasmtime, neither installed on the machine this was written on — CI covers both.

🤖 Generated with Claude Code

janstrakowski and others added 3 commits August 31, 2026 08:34
`SPEC.md` §6 has always said cycles are possible; nothing could construct
one. A `let rec` over a Table literal now can, so a social graph whose
friendships are mutual is expressible without a placeholder type, a `lazy`
keyword, or a second binding form.

The mechanism is evaluation order, not deferral. The Table is created and
bound to the name before any entry runs, so an entry can mention the Table it
belongs to; entries are then evaluated on demand rather than in source order,
so reaching `people.bob` while `.alice` is mid-flight evaluates `.bob` there
and then. That dissolves every dependency between entries that has a
topological order - which is why mutual references need nothing further, and
why a form binding several names at once is still not needed.

What survives is the residual true cycle: `.bob` reaching back into `.alice`,
already in progress and so with no value to give. Only that yields a forward
reference (rec_build.odin), filled the moment `.alice` completes. Storing one
is what makes the back-edge; inspecting one before it is filled is a
genuinely circular definition and fails with a message naming the entry.
Since every entry completes before the `let rec` returns, no finished value
can hold an unfilled one - which is why this is not one of §3's types.

Scope is deliberately narrow, and the docs say so: this builds cyclic
structures, finite graphs with back-edges, not unbounded ones. A stream that
constructs a fresh node per step still recurses to the depth limit. Only a
Table literal written directly as the bound value has entries to reorder;
`let rec x x + 1` still reports "undefined name".

Two things then have to cope with a graph rather than a tree:

- Equality is bisimulation, so two rings of the same shape built by separate
  bindings are the same value - §6 makes equality a question about content,
  and a pointer comparison would answer it wrongly. Assumptions live in a
  union-find with path compression, keeping it near-linear rather than the
  quadratic a visited-pair set would cost. The map is only created when two
  distinct Tables are first assumed equal, so scalar key lookup on the hot
  path of field access still allocates nothing.
- Printing labels the node a back-edge returns to, `#1{n: 1, self: #1}`, so
  output stays finite and the shape is legible. Acyclic values, including a
  sub-Table merely shared between two branches, print exactly as before.

Hashing a cyclic value is refused rather than hung on, alongside the existing
directory-File and Function gaps. §3 pins what a digest encodes, so choosing
a cyclic encoding is a spec decision; `SPEC.md` §6 records what the answer
looks like and `LANGUAGE.md` lists it as unbuilt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the cyclic `let rec` commit, and an audit rather than just its
loose ends: three of these were already wrong before that change.

Already stale:

- §5 said a Table's hash sorts its entries "by key per §6's generic total
  order". It sorts by the key's *digest*, deliberately - §6's cross-type
  ordering is not built, and a digest order is deterministic without it.
- §5 filed hashing, equality and ordering under one heading, which reads as
  though equality were hash-derived. It is structural: entries matched by key
  and compared pairwise. Only `File` compares through its digest, because §3
  defines a File's identity that way. Split into two bullets.
- §8 named the nesting budget's constant `MAX_EVAL_DEPTH`; it has been
  `MAX_NEST_DEPTH` in eval.odin all along.

Stale as of the cyclic `let rec` change:

- §10 stated as a general rule that `rec` "changes which scope the value is
  computed in, and nothing else". True for every shape but the one the same
  section now describes at length, so both the rule and the two consequence
  bullets under it are qualified where they are stated rather than silently
  contradicted further down.
- §10's mutual-recursion bullet explained the one-`rec`-over-a-Table idiom as
  entries being closures over the scope holding the name. That was the whole
  story when only functions could do it. The Table is now bound before its
  entries run and the entries are demanded, which is why the same spelling
  works for data; the closure property still explains why a later call
  resolves, but it is no longer what makes the binding work.
- §8 enumerates failure sources and numbers them, so a circular definition is
  added as the sixth, next to the fourth and before the paragraph that draws
  the conclusion from the list. It is distinguished from the nesting budget:
  detected and named, rather than noticed by running out of stack.
- §15 presented `sha256` as total. It has one gap that is this document's
  rather than an implementation's - a cyclic value, whose digest §6 has not
  settled the encoding of.

Every behavioural claim added here was run against `hb` first: entry order
affecting neither equality nor the digest, the circular-definition message,
`sha256` of a cycle, and an unbounded `let rec` reaching the nesting budget
rather than the circularity check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two unequal Tables could compare equal. Found by testing the case the
original tests missed - cyclic values used as Table *keys*:

  let rec g { .a = { .tag = "a", .next = g.b }, .b = { .tag = "b", .next = g.a } };
  let rec h { .a = { .tag = "a", .next = h.b }, .b = { .tag = "b", .next = h.a } };
  let X { [g.a] = 1, [g.b] = 2, .z = g.a };
  let Y { [h.b] = 2, [h.a] = 1, .z = h.b };
  X == Y        // was true; `g.a == h.b` is false on its own

The bisimulation walk is optimistic: descending into a pair of Tables it
records "assume these two are equal" and compares their entries under that
assumption. Those assumptions are only justified if the descent succeeds.
The header comment claimed nothing needed rolling back because a mismatch
returns false and discards the lot - true of the value spine, where every
caller propagates a false immediately, and false of the one loop that does
not: matching a key scans candidates and keeps going after a failure.

So a failed key match left claims behind. Matching `g.a` against the
candidate `h.b` fails, but not before asserting g.a ~ h.b; the later `.z`
comparison of exactly that pair then short-circuited to true on it.

Key matching is a self-contained question about two subgraphs, so it no
longer shares the walk's state at all - table_find compares each candidate
through values_equal, which makes its own. Isolation costs only the chance to
reuse a valid assumption, and cannot lose an answer: an isolated comparison
re-derives whatever it needs and still terminates on its own back-edges. The
scalar path is untouched, since a Bisim is a stack struct whose map is only
created when two distinct Tables are first assumed equal - and comparing a
scalar key never gets that far.

Five regression tests, covering this and the other gaps the same audit turned
up: different-period cycles that genuinely differ, a cycle against a finite
unrolling, symmetry (the two operands are not handled alike - one is walked,
the other looked up in), and a cyclic value as a key at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@janstrakowski
janstrakowski merged commit 6c216b6 into main Aug 31, 2026
5 checks passed
@janstrakowski
janstrakowski deleted the worktree-cyclic-let-rec branch August 31, 2026 11:42
janstrakowski added a commit that referenced this pull request Aug 31, 2026
#17 landed directory, closure and ctx.cache hashing while this branch was
building the same three as scaffolding for `cached`. Its versions win — they
are merged, they are better factored, and decisively they handle **cyclic
values**, which #16 made constructible and which mine would have recursed on
forever.

Dropped: hash_directory.odin, my hash_function.odin, my hash.odin changes, my
Fs_Entry extension and fs_list_dir_at, my hash tests, and my additions to
examples/hashing.hb (three dedicated hashing examples cover it better).
cache_store.odin now reads directories through main's fs_list_entries_at,
whose getdents walk is better than the /proc/self/fd one it replaces, and
refuses a fifo/socket/device the same way §3's hash does.

Kept, because main has none of it:

  - `cached` itself, rebound to value_digest(v, interp) and Hash_Fail.
  - Four fs write operations — mkdir/rename/unlink/rmdir, on all three
    backends. #17 added reading; the store needs writing.
  - The `#arg`/`#self` coverage, now hash_implicit.odin. main's free_names
    collects identifiers and a uses_ctx flag, but nothing looks at
    Implicit_Name or Hole — they are dynamic lookups no closure captures, so
    `let f func (cached (#arg + 1))` would answer `f 10` with 2.

**Cyclic values are cacheable**, which the format could not previously
express. A Table reached more than once is written `node "N" { … }` at its
first occurrence and `ref "N"` after that; the reader creates each Table
before reading its entries, exactly as §10's evaluation order does, so a
definition always precedes its references and nothing needs patching up. A
restored cycle is bisimulation-equal to the stored one, which is what §6
requires. Labels go on merely shared Tables too — not needed for correctness,
since §6 compares structurally, but it stops a shared value expanding
exponentially on the way out.

230 tests pass; all three targets typecheck.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant